Papers by Ameya Sunil Mahabaleshwarkar
When2Call: When (not) to Call Tools (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing benchmarks focus on the accuracy of tool calling and less on when LMs should (not) call tools. |
| Approach: | They develop a benchmark which evaluates tool-calling decision-making . they leverage multiple-choice nature of the benchmark to develop 'preference optimization' training regime . |
| Outcome: | The proposed benchmark shows that state-of-the-art LMs show room for improvement on When2Call. |